Palo Alto Monitoring Setup Documentation

This document details the hybrid monitoring architecture for a fleet of Palo Alto firewalls managed via Panorama. It outlines how to collect GlobalProtect user counts using the Panorama XML API and interface bandwidth metrics using direct SNMP v3 polling.

Architecture Summary:

1. Panorama API Configuration (User Counts)

We use a custom Python script that reads a configuration file and executes API calls against Panorama, leveraging the target parameter to proxy commands to individual firewalls.

1.1 Configuration File (/conf/panorama_api.conf)

[panorama]
# The IP or Hostname of your Panorama Management Appliance
address = 192.168.161.38
# The API Key generated for your service account
api_key = e6bc89fb5328bd40d3d62c13c5b7369c:T3syDH...

[settings]
request_timeout = 10

[fleet]
# Format: Serial_Number = Location_Name
025401001846 = Alaska
025401001851 = California
025401001850 = Colorado
025401002245 = SouthDakota
025401002210 = Virginia

1.2 Python Script (/conf/read_panorama.py)

Ensure Python 3.9+ and the requests library are installed (pip install requests). Make the script executable: chmod +x /conf/read_panorama.py

(Refer to the codebase for the full Python script. The script parses the configuration, executes API requests to the Panorama instance, and returns JSON-formatted data for Telegraf.)

2. SNMP Configuration (Bandwidth)

Bandwidth (ifTable) metrics are gathered natively by Telegraf using SNMP v3 directly from the firewalls.

2.1 Firewall Setup (via Panorama)

  1. Management Profile: Under Network > Network Profiles > Interface Mgmt, create an Allow-SNMP profile with SNMP checked. Attach this to ethernet1/9 on all firewalls.
  2. SNMP View: Under Device > Setup > Operations > SNMP Setup > Views, create telegraf-view (OID: 1.3.6.1, Option: include, Mask: 0xf0).
  3. SNMP User: Under Device > Setup > Operations > SNMP Setup > V3, create user DOIMONITORv2 using SHA/AES passwords and attach it to telegraf-view.
  4. Commit and Push: Push the template configuration to the firewall fleet.

3. Telegraf Configuration (telegraf.conf)

Add the following inputs to your Telegraf configuration. Restart Telegraf (sudo systemctl restart telegraf) after saving.

#################################
#  User Counts (API Script)     #
#################################
[[inputs.exec]]
  commands = ["python3.9 /conf/read_panorama.py /conf/panorama_api.conf"]
  interval = "300s"
  timeout = "60s"
  data_format = "json"
  json_time_key = "time"
  json_time_format = "unix"
  json_name_key = "name"
  tag_keys = ["location"]

#################################
#  Bandwidth (SNMP v3)          #
#################################
[[inputs.snmp]]
  agents = [
    "udp://192.168.158.39:161", 
    "udp://192.168.159.39:161", 
    "udp://192.168.160.39:161", 
    "udp://192.168.161.39:161", 
    "udp://192.168.162.36:161"
  ]
  version = 3
  sec_name = "DOIMONITORv2"
  auth_protocol = "SHA"
  auth_password = "YOUR_AUTH_PASSWORD"
  sec_level = "authPriv"
  priv_protocol = "AES"
  priv_password = "YOUR_PRIV_PASSWORD"

  [[inputs.snmp.field]]
    oid = "RFC1213-MIB::sysUpTime.0"
    name = "uptime"

  [[inputs.snmp.field]]
    oid = "RFC1213-MIB::sysName.0"
    name = "source"
    is_tag = true

  [[inputs.snmp.table]]
    name = "ifTable"
    inherit_tags = ["source"]

    [[inputs.snmp.table.field]]
      oid = "IF-MIB::ifName"
      name = "ifName"
      is_tag = true

    [[inputs.snmp.table.field]]
      oid = "IF-MIB::ifHCInOctets"
      name = "ifHCInOctets"

    [[inputs.snmp.table.field]]
      oid = "IF-MIB::ifHCOutOctets"
      name = "ifHCOutOctets"

  # VERY IMPORTANT: Only collect metrics for the WAN interface
  [inputs.snmp.tagpass]
    ifName = ["ethernet1/9"]

4. Dashboard Queries (Flux)

Update your frontend JSON/Flux queries to match the new tags and measurements.

4.1 User Counts

|> filter(fn: (r) => r["_measurement"] == "globalprotect_users")
|> group(columns: ["location"])

4.2 Interface Bandwidth (Mbps)

|> filter(fn: (r) => r["_measurement"] == "ifTable")
|> filter(fn: (r) => r["ifName"] == "ethernet1/9")
|> filter(fn: (r) => r["agent_host"] == "192.168.158.39")  // Change IP per widget
|> filter(fn: (r) => r["_field"] == "ifHCInOctets" or r["_field"] == "ifHCOutOctets")
|> derivative(unit: 1s, nonNegative: true)
|> filter(fn: (r) => r._value < 2500000000.0) // 20Gbps limit to drop reboot spikes
|> map(fn: (r) => ({r with _value: r._value * 0.000008})) // Convert to Mbps

5. API Key Management & Regeneration

If the API key expires, is manually revoked, or if the underlying service account password changes, the data collection script will fail and return 0 users. Follow these steps to generate a new permanent key.

Step 1: Generate a New Key

Open a web browser and navigate to the following URL (replace the placeholders with your Panorama IP and service account credentials):

https://192.168.161.38/api/?type=keygen&user=YOUR_USERNAME&password=YOUR_PASSWORD

The browser will return an XML block. Copy the long text string located between the <key>...</key> tags.

Step 2: Update the Configuration File

Open your Telegraf configuration file (/conf/panorama_api.conf) and replace the existing api_key = ... value with the newly generated key. Save the file. Telegraf will automatically use the new key on its next polling interval.

Step 3: Prevent Future Expiration

To ensure the API key remains valid indefinitely:

6. Troubleshooting Guide

Telegraf returns 0 Users / API Key Expired (403 Forbidden)

Symptom: The graphs suddenly drop to 0 for all locations.

Action: Run the debug script (python3 /conf/debug_panorama.py /conf/panorama_api.conf). If Step 2 fails with a 403 Forbidden error, the API key is no longer valid. Follow the steps in Section 5 to regenerate and apply a new key.

Telegraf returns [] or fails to parse

Action: Run the python script manually from the terminal:

python3.9 /conf/read_panorama.py /conf/panorama_api.conf

Check for syntax errors, missing Python modules, or an incorrect path to the config file.

API Script Error 17: "unexpected command"

Action: Verify the target Serial Numbers in panorama_api.conf. Panorama cannot run GlobalProtect commands locally; it requires a valid firewall serial number in the fleet block to proxy the command.

SNMP Connection Timeout

Action: Run an snmpwalk manually from the Telegraf server:

snmpwalk -v3 -l authPriv -u DOIMONITORv2 -a SHA -A 'YOUR_AUTH' -x AES -X 'YOUR_PRIV' 192.168.158.39 1.3.6.1.2.1.1.5.0

If it times out, ensure the Allow-SNMP management profile is attached to ethernet1/9 on the firewall, and that no intermediate security policies are blocking UDP port 161.

Graphs dropping data points during high traffic

Action: Ensure the spike filter in your Flux query isn't too low. A value of 1250000000.0 caps it exactly at 10Gbps, meaning legitimate saturation will be dropped. Raise the filter limit to 2500000000.0 (20Gbps).